nutanix AHV support #3311

tchernomax · 2025-09-26T16:28:10Z

add nutanix AHV support

ignition support nutanix AHV, the only thing to do is to add flatcar.oem.id=nutanix in the kernel cmdline.
However, Nutanix doesn't provide its "Nutanix Guest Tools" in any other format than .deb or .rpm -> so I didn't port it to flatcar.

I managed to create a nutanix AHV flatcar image (without nutanix guest tools... but that doesn't forbid the VM to run) by following this path:

wget https://stable.release.flatcar-linux.net/amd64-usr/current/flatcar_production_image.bin.bz2
bunzip2 flatcar_production_image.bin.bz2
PART=6
LOOP=$(sudo losetup --partscan --find --show flatcar_production_image.bin)
TARGET=$(sudo mktemp -d -p /mnt --suffix -flatcar)
sudo mount "${LOOP}p${PART}" "$TARGET"
echo 'set oem_id="nutanix"' | sudo tee "$TARGET"/grub.cfg
sudo umount "${TARGET}"
sudo rmdir "${TARGET}"
sudo losetup -d "${LOOP}"
qemu-img convert -f raw -O qcow2 flatcar_production_image.bin flatcar_production_image_nutanix.qcow2
rm flatcar_production_image.bin

(it tested it on a Nutanix AHV infra, also tried a flatcar update (via nebraska) -> it works)

And creating the VM with the ignition config provided as "cloud_init" "guest_customization":

...
        "guest_customization": {
          "cloud_init": {
            "user_data": "<base64 ignition conf>"
          }
          "is_overridable": false
        }
...

https://www.nutanix.dev/api_reference/apis/prism_v3.html#tag/vms/paths/~1vms/post

I tried to reproduce this path in this PR, but I can't try the resulting image myself.
It build on my desktop, but I can't upload it to a nutanix AHV platform due to my poor home network connection.
And I failed to build the image on the remote host (where I updated the flatcar_production_image.bin.bz2 official image).

How to use

If what I changed in the code is in fact the equivalent of the adding oem_id="nutanix" in the grub.cfg of the OEM partition.
This PR should be ok.

Otherwise, please guide me (I am a noob in flatcar dev).

Testing done

I created an image :

wget https://stable.release.flatcar-linux.net/amd64-usr/current/flatcar_production_image.bin.bz2
bunzip2 flatcar_production_image.bin.bz2
PART=6
LOOP=$(sudo losetup --partscan --find --show flatcar_production_image.bin)
TARGET=$(sudo mktemp -d -p /mnt --suffix -flatcar)
sudo mount "${LOOP}p${PART}" "$TARGET"
echo 'set oem_id="nutanix"' | sudo tee "$TARGET"/grub.cfg
sudo umount "${TARGET}"
sudo rmdir "${TARGET}"
sudo losetup -d "${LOOP}"
qemu-img convert -f raw -O qcow2 flatcar_production_image.bin flatcar_production_image_nutanix.qcow2
rm flatcar_production_image.bin

upload it to a nutanix ahv cluster
created multiple hosts using this image via terraform restapi provider (the nutanix official provider is currently unable to create VM with ignition configuration as "cloudinit" guest info).
created a k8s cluster on thoses hosts via rke
tried an update via nebraska (flatcar downloaded the new image, I reboot the hosts, it booted on the new version).

I tried this ignition config:

{"ignition":{"version":"3.3.0"},"storage":{"files":[{"path":"/toto","contents":{"compression":"","source":"data:,toto"},"mode":292}]}}

and checked that /toto was actually created at boot.
(I also tried some more advanced ignition config).

CI: http://localhost:8080/job/container/job/packages_all_arches/6848/cldsv/

tormath1 · 2025-09-30T07:43:09Z

Hi @tchernomax and thanks for your contribution, two questions:

About the naming, could we use only nutanix averywhere to match the OEM id? Or _ahv is required because multiple versions of Nutanix images might exist?
However, Nutanix doesn't provide its "Nutanix Guest Tools" in any other format than .deb or .rpm -> so I didn't port it to flatcar.

Do you know if it's open-source? If yes, we can investigate to package this to a Gentoo format.

github-actions · 2025-09-30T07:43:17Z

Build action triggered: https://github.com/flatcar/scripts/actions/runs/19104690012

tchernomax · 2025-10-01T17:40:40Z

* About the naming, could we use only `nutanix` averywhere to match the OEM id? Or `_ahv` is required because multiple versions of Nutanix images might exist?

I don't really know.
I know that nutanix provide an interface which can be plugged on top off VMWare ESX hypervisors (but in that case we use the vmware flatcar image).
There is also nutanix AHV which use a nutanix branded hypervisor (what this MR is for).
While googling I also found nutanix cloud platform and nutanix kubernetes platform but I don't really what it is.

I think the best curse of action would be to ask nutanix themself.
Tomorrow I will ask a colleague (I don't have access myself) to create an issue on nutanix support to ask them to reply on this PR.

(if there is no need for _ahv I will update this PR)

* > However, Nutanix doesn't provide its "Nutanix Guest Tools" in any other format than .deb or .rpm -> so I didn't port it to flatcar.
  Do you know if it's open-source? If yes, we can investigate to package this to a Gentoo format.

I didn't find any source online.
And the content of LICENSE file on hosts where the software is already install suggest it's not.

aurelienlm · 2025-10-02T10:17:22Z

I’ve asked Nutanix Support to take a look at this PR and provide feedback, just in case it helps

aurelienlm · 2025-10-02T14:07:21Z

the answer :
If i understand correctly, you are trying to integrate nutanix guest tools in flatcar ?

1 - Nutanix guest tools isn't opensource.
2 - I will check internally whether we can raise an internal request to our engineering for this

I will provide an update tomorrow or Monday on this.

tormath1 · 2025-10-07T08:31:31Z

the answer : If i understand correctly, you are trying to integrate nutanix guest tools in flatcar ?

1 - Nutanix guest tools isn't opensource. 2 - I will check internally whether we can raise an internal request to our engineering for this

I will provide an update tomorrow or Monday on this.

Great thanks for asking. That'd be nice to have those guest tools to get the full support. Do you know what they add? I guess it's some helpers for network, system information, etc. ?

Do you folks think that'd make sense to wait for this tools or Flatcar can be used without on Nutanix?

tchernomax · 2025-10-07T08:46:02Z

Do you know what they add? I guess it's some helpers for network, system information, etc. ?

From what I understand (@nutanix please correct me if I am wrong) it's for sending system information from the VM to the host (like ip and such).

Do you folks think that'd make sense to wait for this tools or Flatcar can be used without on Nutanix?

We plan on using flatcar without nutanix tools, from the nutanix support told us it's not ideal but the system/flatcar should be fine.
(though we only tests flatcar on nutanix (without tools) on a simple POC)

tormath1 · 2025-10-07T09:39:15Z

Do you know what they add? I guess it's some helpers for network, system information, etc. ?

From what I understand (@nutanix please correct me if I am wrong) it's for sending system information from the VM to the host (like ip and such).

Do you folks think that'd make sense to wait for this tools or Flatcar can be used without on Nutanix?

We plan on using flatcar without nutanix tools, from the nutanix support told us it's not ideal but the system/flatcar should be fine. (though we only tests flatcar on nutanix (without tools) on a simple POC)

Ok then, we can start with an image without guest tools and revisit this later.

To get back on the first point (nutanix_ahv vs nutanix) that'd be nice to know which one we can use, to have some consistency with the OEM ID. Upstream Ignition is using nutanix so I'd be in favor of using nutanix everywhere.

aurelienlm · 2025-10-07T14:15:54Z

sorry the Nutanix support team can't help us on this. They have close the case.

I have reach out to our account team

tchernomax · 2025-10-08T08:09:26Z

Upstream Ignition is using nutanix so I'd be in favor of using nutanix everywhere.

I changed this PR to remove "AHV" reference and only use nutanix.

tormath1 · 2025-10-08T14:35:06Z

Upstream Ignition is using nutanix so I'd be in favor of using nutanix everywhere.

I changed this PR to remove "AHV" reference and only use nutanix.

Thanks! I'll setup our CI system to build nutanix images then. Once the image will be built, will you be available for running test for it?

tchernomax · 2025-10-09T14:01:49Z

Once the image will be built, will you be available for running test for it?

yes (I might not be very responsive but I will make time to tests the images you provide)

tchernomax · 2025-10-21T12:03:57Z

I'll setup our CI system to build nutanix images then

@tormath1 any news ?

tormath1 · 2025-10-22T08:14:55Z

I'll setup our CI system to build nutanix images then

@tormath1 any news ?

@tchernomax thanks for the heads-up! I started a build, images should be available here: https://bincache.flatcar-linux.net/images/amd64/9999.0.100+tormath1-nutanix/ in a few hours.

EDIT: Now available: https://bincache.flatcar-linux.net/images/amd64/9999.0.100+tormath1-nutanix/flatcar_production_nutanix_image.img.bz2

tchernomax · 2025-10-28T09:35:02Z

EDIT: Now available: https://bincache.flatcar-linux.net/images/amd64/9999.0.100+tormath1-nutanix/flatcar_production_nutanix_image.img.bz2

@tormath1
Sorry for the delay.

I just tried the image https://bincache.flatcar-linux.net/images/amd64/9999.0.100+tormath1-nutanix/flatcar_production_nutanix_image.img.bz2
with this butane configuration:

	variant: flatcar
	version: 1.0.0
	kernel_arguments:
	  should_exist:
	    - flatcar.autologin
	storage:
	  files:
	    - path: /toto
	      overwrite: true
	      mode: 0644
	      contents:
		inline: toto

And it's working properly.

I am autologged and the file /toto is created.

No service in "failing" state.
Nothing special in the journal or dmesg.

Everything seems fine.

localhost ~ # tail -n+l1 /etc/*relea*

==> /etc/lsb-release <==
DISTRIB_ID="Flatcar Container Linux by Kinvolk"
DISTRIB_RELEASE=9999.0.100+tormath1-nutanix
DISTRIB_CODENAME="Oklo"
DISTRIB_DESCRIPTION="Flatcar Container Linux by Kinvolk 9999.0.100+tormath1-nutanix (Oklo)"

==> /etc/os-release <==
NAME="Flatcar Container Linux by Kinvolk"
ID=flatcar
ID_LIKE=coreos
VERSION=9999.0.100+tormath1-nutanix
VERSION_ID=9999.0.100
BUILD_ID=tormath1-nutanix
SYSEXT_LEVEL=1.0
PRETTY_NAME="Flatcar Container Linux by Kinvolk 9999.0.100+tormath1-nutanix (Oklo)"
ANSI_COLOR="38;5;75"
HOME_URL="https://flatcar.org/"
BUG_REPORT_URL="https://issues.flatcar.org"
FLATCAR_BOARD="‘amd64-usr"
CPE_NAME="cpe:2.3:o:flatcar-linux:flatcar_linux:9999.0.100+tormath1-nutanix:*:*:*:*:*:*:*"

Regards

tchernomax · 2025-11-04T15:22:48Z

@tormath1 any news ?

tormath1

LGTM, we can just add one last thing: a changelog entry to be added (similar to https://github.com/flatcar/scripts/blob/main/changelog/changes/2025-06-30-stackit.md)

As this is a new platform, I think we can directly provide images for all channels (alpha, beta, stable, LTS)

tchernomax · 2025-11-05T09:37:17Z

we can just add one last thing: a changelog entry to be added (similar to https://github.com/flatcar/scripts/blob/main/changelog/changes/2025-06-30-stackit.md)

I just amend the commit to add changelog/changes/2025-11-05-nutanix.md.

As this is a new platform, I think we can directly provide images for all channels (alpha, beta, stable, LTS)

I will let you be judge on that.

tormath1 · 2025-11-05T09:40:41Z

Two things:

as part of a CNCF project, all the commits must be signed-off (see instructions: https://github.com/flatcar/scripts/pull/3311/checks?check_run_id=54561187192)
next week there is the monthly Flatcar office hours, would you be interested to present your work and a quick demo of Nutanix support? (Agenda for the Flatcar Office Hours on Wednesday, 12th of November 2025, 2:30pm UTC Flatcar#1907)

Signed-off-by: Maxime de Roucy <[email protected]>

tchernomax · 2025-11-05T10:02:48Z

* as part of a CNCF project, all the commits must be signed-off (see instructions: https://github.com/flatcar/scripts/pull/3311/checks?check_run_id=54561187192)

done

* next week there is the monthly Flatcar office hours, would you be interested to present your work and a quick demo of Nutanix support? ([Agenda for the Flatcar Office Hours on Wednesday, 12th of November 2025, 2:30pm UTC Flatcar#1907](https://github.com/flatcar/Flatcar/discussions/1907))

wow… I didn't expected that one…
First : I'm not very confident with my spoken English (I can manage, but I have a tendency to look for my words)
Second: I already have an appointment 14:00→15:30 that day.

If you think the first point shouldn't be a problem, I can ask my boss if he could move my appointment and free me for the "Flatcar Office Hours".

tchernomax · 2025-11-05T10:29:13Z

I can ask my boss if he could move my appointment and free me for the "Flatcar Office Hours".

I just asked and it's ok → I am free next wednesday 14:30 UTC.

So if you think my spoken English shouldn't be a problem I can do a quick presentation of the new nutanix image (dev + live demo).

tormath1 · 2025-11-05T14:00:45Z

Cherry-picked to:

flatcar-4487
flatcar-4459
flatcar-4230

tormath1 · 2025-11-05T14:07:51Z

@tchernomax many thanks for the contribution, this will be available in the next set of releases (alpha, beta and stable) around next week.

For the Office Hours, it's really open to contributors to show case this kind of contribution. There are usually around 8/9 contributors / maintainers and there's nothing formal (you can prepare some slides or just share your screen without support, you decide!)
We can discuss the next steps (e.g write some documentation to use Flatcar on Nutanix) as well.

If you're OK to join, feel free to drop a comment in the discussion to mention a Flatcar / Nutanix demo: flatcar/Flatcar#1907

tchernomax requested a review from a team as a code owner September 26, 2025 16:28

tormath1 added the main label Sep 30, 2025

tormath1 added this to Flatcar tactical, release planning, and roadmap Sep 30, 2025

tormath1 moved this to ✅ Testing / in Review in Flatcar tactical, release planning, and roadmap Sep 30, 2025

tchernomax force-pushed the nuta branch from ee0606c to 28f7b9e Compare October 8, 2025 08:07

github-actions bot mentioned this pull request Oct 22, 2025

Monthly contributions report 2025-09-22 - 2025-10-21 flatcar/Flatcar#1927

Open

tormath1 approved these changes Nov 5, 2025

View reviewed changes

tormath1 added alpha lts beta stable labels Nov 5, 2025

tchernomax force-pushed the nuta branch from 28f7b9e to 6810747 Compare November 5, 2025 09:35

nutanix AHV support

87a1946

Signed-off-by: Maxime de Roucy <[email protected]>

tchernomax force-pushed the nuta branch from 6810747 to 87a1946 Compare November 5, 2025 09:49

tormath1 merged commit aee197d into flatcar:main Nov 5, 2025
4 checks passed

github-project-automation bot moved this from ✅ Testing / in Review to Implemented in Flatcar tactical, release planning, and roadmap Nov 5, 2025

tormath1 removed the lts label Nov 5, 2025

nutanix AHV support #3311

nutanix AHV support #3311

Conversation

tchernomax commented Sep 26, 2025 • edited by tormath1 Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

add nutanix AHV support

How to use

Testing done

Uh oh!

tormath1 commented Sep 30, 2025

Uh oh!

github-actions bot commented Sep 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

tchernomax commented Oct 1, 2025

Uh oh!

aurelienlm commented Oct 2, 2025

Uh oh!

aurelienlm commented Oct 2, 2025

Uh oh!

tormath1 commented Oct 7, 2025

Uh oh!

tchernomax commented Oct 7, 2025

Uh oh!

tormath1 commented Oct 7, 2025

Uh oh!

aurelienlm commented Oct 7, 2025

Uh oh!

tchernomax commented Oct 8, 2025

Uh oh!

tormath1 commented Oct 8, 2025

Uh oh!

tchernomax commented Oct 9, 2025

Uh oh!

tchernomax commented Oct 21, 2025

Uh oh!

tormath1 commented Oct 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

tchernomax commented Oct 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

tchernomax commented Nov 4, 2025

Uh oh!

tormath1 left a comment

Choose a reason for hiding this comment

Uh oh!

tchernomax commented Nov 5, 2025

Uh oh!

tormath1 commented Nov 5, 2025

Uh oh!

tchernomax commented Nov 5, 2025

Uh oh!

tchernomax commented Nov 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

tormath1 commented Nov 5, 2025

Uh oh!

tormath1 commented Nov 5, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

tchernomax commented Sep 26, 2025 •

edited by tormath1

Loading

github-actions bot commented Sep 30, 2025 •

edited

Loading

tormath1 commented Oct 22, 2025 •

edited

Loading

tchernomax commented Oct 28, 2025 •

edited

Loading

tchernomax commented Nov 5, 2025 •

edited

Loading